AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time	single_prediction_time
	1_Linear	Linear	auc	0.974028	39.39	0.0305
	2_Default_LightGBM	LightGBM	auc	0.979291	28.3	0.0166
	3_Default_Xgboost	Xgboost	auc	0.973694	27.25	0.0522
	4_Default_CatBoost	CatBoost	auc	0.985145	29.47	0.032
	5_Default_NeuralNetwork	Neural Network	auc	0.969814	25.24	0.0436
	6_Default_RandomForest	Random Forest	auc	0.973103	34.72	0.4096
	11_LightGBM	LightGBM	auc	0.980817	27.84	0.0151
	7_Xgboost	Xgboost	auc	0.977909	34.37	0.0211
	15_CatBoost	CatBoost	auc	0.988635	28.05	0.0286
	19_RandomForest	Random Forest	auc	0.984726	33.73	0.2163
	23_NeuralNetwork	Neural Network	auc	0.976917	24.98	0.0343
	12_LightGBM	LightGBM	auc	0.97603	30.46	0.0146
	8_Xgboost	Xgboost	auc	0.974095	28.99	0.0154
	16_CatBoost	CatBoost	auc	0.981064	27.86	0.024
	20_RandomForest	Random Forest	auc	0.980073	33.44	0.1226
	24_NeuralNetwork	Neural Network	auc	0.974066	30.74	0.0395
	13_LightGBM	LightGBM	auc	0.981694	34.59	0.0144
	9_Xgboost	Xgboost	auc	0.92502	26.99	0.0215
	17_CatBoost	CatBoost	auc	0.977537	25.68	0.0187
	21_RandomForest	Random Forest	auc	0.963807	32.83	0.147
	25_NeuralNetwork	Neural Network	auc	0.966877	28.99	0.0331
	14_LightGBM	LightGBM	auc	0.978757	30.05	0.0191
	10_Xgboost	Xgboost	auc	0.500868	29.36	0.0173
	18_CatBoost	CatBoost	auc	0.985651	41.41	0.024
	22_RandomForest	Random Forest	auc	0.969623	35.12	0.2364
	26_NeuralNetwork	Neural Network	auc	0.979119	25.98	0.0387
	15_CatBoost_GoldenFeatures	CatBoost	auc	0.986785	34.02	0.0374
	18_CatBoost_GoldenFeatures	CatBoost	auc	0.985174	31.45	0.0385
	4_Default_CatBoost_GoldenFeatures	CatBoost	auc	0.986251	32	0.0494
	27_CatBoost	CatBoost	auc	0.978795	34.5	0.0844
	28_CatBoost	CatBoost	auc	0.989007	29.75	0.0192
	29_CatBoost_GoldenFeatures	CatBoost	auc	0.981675	26.7	0.0404
	30_CatBoost_GoldenFeatures	CatBoost	auc	0.986966	26.06	0.037
	31_RandomForest	Random Forest	auc	0.983591	27.86	0.175
	32_RandomForest	Random Forest	auc	0.98014	29.42	0.1273
	33_LightGBM	LightGBM	auc	0.982275	25.12	0.0123
	34_LightGBM	LightGBM	auc	0.982628	25.38	0.011
	35_LightGBM	LightGBM	auc	0.981865	25.48	0.0112
	36_LightGBM	LightGBM	auc	0.980979	25.31	0.0144
	37_RandomForest	Random Forest	auc	0.983944	29.01	0.1524
	38_RandomForest	Random Forest	auc	0.97604	28.58	0.131
	39_NeuralNetwork	Neural Network	auc	0.973742	24.81	0.0278
	40_NeuralNetwork	Neural Network	auc	0.974829	25.08	0.0307
	41_Xgboost	Xgboost	auc	0.977212	25.37	0.0142
	42_Xgboost	Xgboost	auc	0.982113	25.04	0.0125
	43_NeuralNetwork	Neural Network	auc	0.982418	25.26	0.0337
	44_CatBoost	CatBoost	auc	0.980674	25.63	0.0148
	45_CatBoost	CatBoost	auc	0.989035	26.76	0.0188
	46_CatBoost	CatBoost	auc	0.977537	28.33	0.016
	47_CatBoost	CatBoost	auc	0.986251	27.24	0.0216
	48_RandomForest	Random Forest	auc	0.983372	30.78	0.1675
the best	Ensemble	Ensemble	auc	0.992048	7.78	0.1767

	score	threshold
logloss	0.693206	nan
auc	0.500868	nan
f1	0.666667	0.452935
accuracy	0.502183	0.505075
precision	0.501818	0.505075
recall	1	0.452935
mcc	0.00445767	0.505075

	score	threshold
logloss	0.693206	nan
auc	0.500868	nan
f1	0.547619	0.505075
accuracy	0.502183	0.505075
precision	0.501818	0.505075
recall	0.60262	0.505075
mcc	0.00445767	0.505075

	Predicted as 0	Predicted as 1
Labeled as 0	92	137
Labeled as 1	91	138

	score	threshold
logloss	0.200862	nan
auc	0.980817	nan
f1	0.955556	0.556621
accuracy	0.956332	0.556621
precision	1	0.928601
recall	1	0.00397278
mcc	0.913221	0.556621

	score	threshold
logloss	0.200862	nan
auc	0.980817	nan
f1	0.955556	0.556621
accuracy	0.956332	0.556621
precision	0.972851	0.556621
recall	0.938865	0.556621
mcc	0.913221	0.556621

	score	threshold
logloss	0.196503	nan
auc	0.97603	nan
f1	0.955947	0.597856
accuracy	0.956332	0.597856
precision	0.992701	0.99474
recall	1	3.20726e-08
mcc	0.913221	0.734665

	score	threshold
logloss	0.1889	nan
auc	0.981694	nan
f1	0.958785	0.452312
accuracy	0.958515	0.452312
precision	1	0.927225
recall	1	2.76086e-05
mcc	0.917109	0.452312

	score	threshold
logloss	0.160448	nan
auc	0.978757	nan
f1	0.964758	0.523377
accuracy	0.965066	0.523377
precision	1	0.986192
recall	1	0.00049015
mcc	0.930273	0.523377

	score	threshold
logloss	0.193329	nan
auc	0.988635	nan
f1	0.965368	0.293846
accuracy	0.965066	0.293846
precision	1	0.816171
recall	1	0.0198913
mcc	0.930273	0.293846

	score	threshold
logloss	0.175616	nan
auc	0.986785	nan
f1	0.961039	0.334079
accuracy	0.960699	0.334079
precision	1	0.80993
recall	1	0.00460704
mcc	0.921538	0.334079

	score	threshold
logloss	0.275119	nan
auc	0.981064	nan
f1	0.955947	0.458986
accuracy	0.956332	0.458986
precision	1	0.694066
recall	1	0.0120018
mcc	0.913919	0.509014

	score	threshold
logloss	0.235786	nan
auc	0.977537	nan
f1	0.960352	0.482843
accuracy	0.960699	0.482843
precision	1	0.955013
recall	1	0.000954883
mcc	0.921538	0.482843

	score	threshold
logloss	0.180536	nan
auc	0.985651	nan
f1	0.964758	0.451991
accuracy	0.965066	0.451991
precision	1	0.899215
recall	1	0.000418861
mcc	0.930273	0.451991

	score	threshold
logloss	0.145194	nan
auc	0.985174	nan
f1	0.960699	0.271299
accuracy	0.960699	0.271299
precision	1	0.977599
recall	1	0.000405224
mcc	0.921397	0.271299

	score	threshold
logloss	0.206201	nan
auc	0.984726	nan
f1	0.955947	0.522409
accuracy	0.956332	0.522409
precision	1	0.85675
recall	1	0.0194263
mcc	0.913221	0.56226

	score	threshold
logloss	0.192042	nan
auc	0.974028	nan
f1	0.941176	0.538017
accuracy	0.943231	0.538017
precision	0.976526	0.538017
recall	0.908297	0.538017
mcc	0.888635	0.538017

feature	Learner_1	Learner_2	Learner_3	Learner_4	Learner_5
PEER_PRESSURE	2.09855	1.97751	2.10596	2.24214	2.11845
ALLERGY	1.71851	1.8115	1.80494	1.6434	1.99987
CHRONIC DISEASE	1.87657	1.63242	1.72145	1.71654	1.80335
YELLOW_FINGERS	1.60769	1.59401	1.57961	1.70178	1.69653
SWALLOWING DIFFICULTY	1.54115	1.39416	1.52935	1.57504	1.50732
COUGHING	1.6269	1.44695	1.61924	1.30983	1.45889
WHEEZING	1.39098	1.33944	1.35351	1.6218	1.23305
ALCOHOL CONSUMING	1.2172	1.51097	1.01974	1.43756	1.6208
FATIGUE	1.22949	1.23666	1.24562	1.05542	1.27239
ANXIETY	0.676465	1.11655	0.820949	1.03973	0.927658
CHEST PAIN	0.451082	0.609851	0.301484	0.389733	0.68452
SMOKING	0.404751	0.352542	0.398707	0.489795	0.317827
SHORTNESS OF BREATH	0.222579	0.360146	0.254631	0.399099	0.113896
AGE	-0.0683001	0.0651653	0.0293103	-0.0196802	0.14053
GENDER	-0.0705105	-0.180772	-0.240995	-0.128562	0.147742
intercept	-4.88958	-5.06286	-4.78165	-5.16035	-5.28215

	score	threshold
logloss	0.234776	nan
auc	0.980073	nan
f1	0.929032	0.527048
accuracy	0.930131	0.579328
precision	1	0.661859
recall	1	0
mcc	0.863562	0.610337

	score	threshold
logloss	0.287266	nan
auc	0.963807	nan
f1	0.902386	0.594454
accuracy	0.901747	0.594454
precision	1	0.73422
recall	1	0.0325234
mcc	0.803562	0.594454

	score	threshold
logloss	0.287379	nan
auc	0.969623	nan
f1	0.913684	0.59222
accuracy	0.91048	0.59222
precision	1	0.80151
recall	1	0
mcc	0.823232	0.59222

	score	threshold
logloss	0.232964	nan
auc	0.976917	nan
f1	0.944321	0.492837
accuracy	0.945415	0.492837
precision	1	0.99338
recall	1	6.41845e-12
mcc	0.891518	0.492837

	score	threshold
logloss	0.192402	nan
auc	0.974066	nan
f1	0.944321	0.509772
accuracy	0.945415	0.509772
precision	1	0.999026
recall	1	0.000268946
mcc	0.891518	0.509772

	score	threshold
logloss	0.244839	nan
auc	0.966877	nan
f1	0.938865	0.505872
accuracy	0.938865	0.505872
precision	1	0.991168
recall	1	4.76261e-06
mcc	0.877863	0.551479

	score	threshold
logloss	0.190663	nan
auc	0.979119	nan
f1	0.947598	0.46477
accuracy	0.947598	0.46477
precision	1	0.999892
recall	1	4.76882e-06
mcc	0.895197	0.46477

	score	threshold
logloss	0.329685	nan
auc	0.978795	nan
f1	0.95671	0.457538
accuracy	0.956332	0.457538
precision	1	0.860599
recall	1	0.00718024
mcc	0.913221	0.505393

	score	threshold
logloss	0.414974	nan
auc	0.989007	nan
f1	0.969298	0.478908
accuracy	0.969432	0.478908
precision	1	0.604072
recall	1	0.100235
mcc	0.939008	0.484481

AutoML Leaderboard

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 10_Xgboost

Extreme Gradient Boosting (Xgboost)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.505075)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 11_LightGBM

LightGBM

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.556621)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 12_LightGBM

LightGBM

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.597856)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 13_LightGBM

LightGBM

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.452312)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 14_LightGBM

LightGBM

Validation

	score	threshold
logloss	0.164462	nan
auc	0.981675	nan
f1	0.956522	0.361882
accuracy	0.956332	0.361882
precision	1	0.9133
recall	1	0.00146918
mcc	0.912803	0.407268

	score	threshold
logloss	0.204195	nan
auc	0.979291	nan
f1	0.955947	0.487291
accuracy	0.956332	0.487291
precision	1	0.913798
recall	1	0.000282286
mcc	0.912803	0.487291

	score	threshold
logloss	0.157187	nan
auc	0.986966	nan
f1	0.956332	0.368843
accuracy	0.956332	0.368843
precision	1	0.862884
recall	1	0.00186637
mcc	0.912803	0.437405

	score	threshold
logloss	0.193155	nan
auc	0.983591	nan
f1	0.956332	0.474051
accuracy	0.956332	0.474051
precision	1	0.903664
recall	1	0.0198381
mcc	0.912664	0.474051

	score	threshold
logloss	0.242589	nan
auc	0.98014	nan
f1	0.945946	0.579601
accuracy	0.947598	0.579601
precision	1	0.783775
recall	1	0.0117756
mcc	0.896874	0.579601

	score	threshold
logloss	0.224591	nan
auc	0.982275	nan
f1	0.956332	0.486919
accuracy	0.956332	0.486919
precision	1	0.824257
recall	1	0.000316717
mcc	0.913221	0.527515

	score	threshold
logloss	0.200515	nan
auc	0.982628	nan
f1	0.960699	0.469438
accuracy	0.960699	0.469438
precision	1	0.925408
recall	1	0.000141272
mcc	0.921397	0.469438

	score	threshold
logloss	0.185302	nan
auc	0.981865	nan
f1	0.951965	0.458253
accuracy	0.951965	0.458253
precision	1	0.921568
recall	1	0.00311882
mcc	0.904068	0.503278

	score	threshold
logloss	0.19414	nan
auc	0.980979	nan
f1	0.951327	0.533745
accuracy	0.951965	0.533745
precision	1	0.913974
recall	1	0.00207946
mcc	0.904241	0.533745

	score	threshold
logloss	0.208765	nan
auc	0.983944	nan
f1	0.954955	0.574657
accuracy	0.956332	0.574657
precision	1	0.784506
recall	1	0
mcc	0.914374	0.574657